New ideas are getting harder to find — and more expensive

It’s an age of astonishing technological progress — but are we starting to have a harder time coming up with new ideas?

Yes, argues a group of MIT Sloan and Stanford University researchers, who found in a study published by the National Bureau of Economic Research in March that the productivity of scientific research is falling sharply across the board.

That, they argue, is because researchers are putting in more and more effort to sustain the same — or even a slightly lower — pace of idea generation as we experienced half a century ago.

“Just to sustain the constant growth in GDP per person, the U.S. must double the amount of research effort put into searching for a new idea every 13 years to offset the increased difficulty in finding new ideas,” write MIT Sloan professor of applied economics John Van Reenen, Stanford University professors Nicholas Bloom and Charles I. Jones, and Stanford doctoral candidate Michael Webb.

Moore’s Law — the observed doubling of the number of transistors packed onto new computer central processing units every two years — stands as a prime example. The doubling effect represents a growth rate of 35 percent each year, and that growth is driven only by ever-more-extensive research, the authors write.

“Many commentators note that Moore’s Law is not a law of nature, but instead results from intense research effort: Doubling the transistor density is often viewed as a goal or target for research programs,” they write.

They continue: “The constant exponential growth implied by Moore’s Law has been achieved only by a massive increase in the amount of resources devoted to pushing the frontier forward.”

In fact, research efforts toward semiconductor improvement have risen by a factor of 18 since the early 1970s, the study found, while productivity has fallen by the same factor. Taken together, that means it’s about 18 times harder today to push Moore’s Law forward than it was half a century ago, the authors write.

The same trend held true for agricultural crop yields. Per-acre yields for corn, soybeans, wheat, and cotton grew about 1.5 percent on average every five years between 1960 and 2015, while the number of researchers working on boosting yields has risen by a factor of between three and 25, depending on the crop.

“It is … evident … that research productivity has fallen sharply for agricultural yields,” the authors write. “Yield growth is relatively stable or even declining, while the effective research that has driven this yield growth has risen tremendously.”

And when the researchers examined the pharmaceutical industry, it was the same story: Research effort put into discovering new drugs since the 1970s rose by 6 percent each year, while the productivity of those efforts, measured by how many were approved by the federal Food and Drug Administration, fell by 3.5 percent per year.

When comparing the years of life per 100 people saved by research targeting cancer since the 1970s to the number of medical studies published over the same time period, the study found that productivity declined by a factor of 1.2 for all work, and a factor of 4.8 for clinical trials.But cancer research productivity actually rose between 1975 and the mid 1980s, the authors note. “These cases suggest that it may get easier to find new ideas at first before getting harder, at least in some areas.”

x x

In the broader economy, the study found it now takes about 15 times as many researchers today versus 30 years ago for a firm to enjoy the same rate of revenue growth.

To keep afloat, investment is key

Van Reenen said one factor that could explain the trend is that it’s simply taking longer for researchers to achieve the level of education they need to make a breakthrough discovery.

“As the total amount of knowledge becomes larger and larger and larger, it becomes increasingly difficult to get to its frontier of that knowledge,” Van Reenen said. “It was much easier a couple thousand years ago.”

Narrowing the focus of one’s studies to specialize in a very particular domain has emerged as a common workaround to that problem, Van Reenen said, but that strategy breeds its own set of issues.

“In order to carry on innovating, you’re constantly working together, and it’s very complicated to get all of these people and ideas together,” he said. “That, itself, could be a reason why things start slowing down.”

Van Reenen said any sort of hard limits to technological growth is a long way off, and population growth, the increasing ease of connection and communication, and globalization present new opportunities for new ideas to emerge.

“As long as we keep increasing the amount of resources we put into research, we’ll keep doing that,” he said. But that requires continuous investment of national GDP, and he worries the impetus to do that hasn’t been as strong lately.

“The concern I have is that the investments are not being done. I think a lot of the time you hear we’ll just cut the top tax rates to generate lots of innovation — I’m pretty skeptical about that in terms of the incentives you’re going to give,” he said. 

Instead, investment in new technologies and innovation should be more targeted, with governments directly allocating funds worthwhile projects. That could cover funding for specific projects, or expanding science, technology, engineering, and mathematics opportunities for women and people of color, who are underrepresented in those fields.

“Both have positive effects on growth and equality. Instead of giving away $5 trillion in tax cuts, use that to invest in growth opportunities for the future,” said Van Reenen.